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THE CONCLUSIONS OF AM EARLIER STUDY LED TO THIS STUDY OF 
THE EFFECTS OF QUESTIONS INTERSPERSED IN THE READING 
MATERIAL, OF QUESTIONS PRESENTED AFTER THE MATERIAL, AND OF 
THE MODE OF QUESTIONING ON LEARNING FROM PROSE. ABOUT 120 
PSYCHOLOGY STUDENTS READ TWENTY 10-SENTENCE PARAGRAPHS OF 
BIOGRAPHICAL NATURE WITH EITHER A QUESTION BEFORE EACH 
PARAGRAPH, TWO OR MORE QUESTIONS AFTER TWO OR MORE 
PARAGRAPHS, OR WITH QUESTIONS APPROPRIATELY INTERSPERSED IN 
THE WHOLE MATERIAL. NO KNOWLEDGE OF THE RESULTS WAS GIVEN 
AFTER THE QUESTIONS, AND THE STUDENTS WERE NOT ALLOWED TO 
REVIEW THE PARAGRAPHS. THE POST-TEST WAS A MULTIPLE -CHOICE 
TEST CONSISTING OF THE 20 QUESTIONS FOUND IN THE MATERIAL 
READ AND OF 20 NEW QUESTIONS TO TEST INCIDENTAL LEARNING. 
RESULTS SUPPORT EARLIER FINDINGS THAT POST-QUESTIONING 
PRODUCES HIGHER LEARNING, PARTICULARLY WHEN THE PACING OF 
QUESTIONS IS INCREASED. THE HIGHER THE PACING OF QUESTIONS, 
THE LARGER THE DISCRIMINATION BETWEEN RELEVANT AND INCIDENTAL 
RECALL. TOO MUCH EXPOSURE TO THE QUESTIONS DECREASED 
INCIDENTAL RECALL. REGARDLESS OF PACING OR THE LOCATION OF 
QUESTIONS, HIGHER INCIDENTAL RECALL WAS OBTAINED WHEN THE 
INCIDENTAL TEXT MATERIAL FOLLOWED THE RELEVANT TEXT MATERIAL. 
THE MODE OF QUESTIONING DID NOT INFLUENCE LEARNING. A GRAPHIC 
REPRESENTATION OF FINDINGS IS INCLUDED. THIS PAPER WAS 
PRESENTED AT THE AMERICAN EDUCATIONAL RESEARCH ASSOCIATION 
CONFERENCE (CHICAGO, FEBRUARY 6-10, 1968). (MS) 
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I *m sure we are all aware of various efforts to 
apply psychological techniQues and theories to education. 
Certain principles from the learning laboratory, most nota- 
bly operant conditioning principles, have been extended 
into education in the form of programmed instruction. 
Although there may be disagreement about whether these 
principles are actually represented in such learning pro- 
grams, it seems clear that the beginnings of a general tech- 
nology of Instruction are evolving out of the research which 
such materials have stimulated. 

But there has come about a certain research 
dilemma concerning programmed insti*uetion, it has been 
found that learning can take place efficiently without the 
program format. For instance, college students can learn 
equally well in less time from ordinary prose or mere summa- 
ries than they do with the programmed material covering the 
same information. 
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^ Paper read at the Annual Convention of the AERA - Chicago, 

1968. ' * 
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So part of the current Instructional theory and 

research Is beginning to concern Itself with the develop- 

( 

ment of principles and techniques which relate especially 
to the mature; student who probably does not need to be led 
step by step through a certain content area. The focus of 
this aspect of instructional technology concerns itself 
with the stimulus controls for general problem solving 
responses which students already possess (raathemagenlc 
behaviors, as Rothkopf has called them). The emphasis here 
is not upon the development of particular skills, but upon 
the ways in which available skills might be elicited in the 
presence of printed materials which themselves are not pro- 
grammed . 

Elaboration of the technology relevant to the 
mature learner has important practical, as well as theo- 
retical, implications. Through the use of questions or 
various other cues, it might be possible to program learning 
from currently available text materials. 

Problem 

This study was an attempt to extend the results 
of earlier work which showed that learning from prose 
materials can be improved If questions are included after 
seqments of the material. Such improvement does not occur 
if the questions are placed before those segments. There 
were some hints in the earlier research which indicated that 

condition for this facilitation might be that 



a necessary 



the post-questions need to occur fairly often in conjunction 
with the material to which they relate. That is, if 10 



post-questions are interspersed in the material, facilita- 
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reading, facilitation will not occur. The earlier research 
also suggested that Increased pacing of pre-questions might 
actually lead to depressed performance. 



Method and Results 



I won*t burden you with a lengthy description of 
the study. The design is indicated on the first page of 
the Appendix. Briefly, the subjects (Ss) read a 2000 word 
biographical passage. Some of the ^s saw a question before 
each 10 sentence paragraph, some saw two questions after 
two 10 sentence paragraphs, etc. Other Ss saw these 
questions, appropriately spaced, after the paragraphs. No 
knowledge of results was given after the questions, and Ss 
were not allowed to review the paragraphs once they had 
turned over the page. All questions and all paragraphs were 
on separate sheets of paper. After reading all 20 para- 
graphs and questions, ^s immediately completed a 40 item 
multiple-choice test which included the 20 questions they 
had seen with the text and 20 other questions which they 
had not seen (the latter were used to test incidental 
learning). For half of the Ss the questions in the text 
were multiple-choice, for the other half the questions were 



constructed response items. 











Figure 1 (in the Appendix) Indicates some of the 
results. It Is clear, for Instance, that questions after 
the paragraphs produced higher learning than questions 
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before (p < .001). This repiica-cea une x- 
earlier studies. Figure 1 also indicates that there was 
an Interaction between the pacing and position of the 
questions (£ < .05). As the pacing of the questions 
Increased the post-question groups did better, the pre- 



question groups did worse. 

There was no Interaction between question position, 

pacing, and the retention of relevant or incidental content; 

hence, the pacing effect was not due to any focussing upon 

the relevant material mainly by the groups which saw the 

questions before the paragraphs. 

Figure 2 shows the focussing effect, which occurred 

across all groups. The higher the pacing of the questions 
the larger the discrimination between relevant and Incidental 
recall (£ < .05) . It Is Interesting to note that frequent 
exposure to the questions did not produce a sharp rise In 
recall of the relevant material., but rather a sharp relative 
decrease In recall of the Incidental material. There was, 
of course, an overall higher level of recall for the rele 

vant material than for the Incidental (£ < .001). 

I will mention two other results In passing. First, 
as can be seen In Figure 3, regardless of pacing or location 
of the questions, higher Incidental recall was obtained when 
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the Incidental text material followed the relevant text 
material (interaction £ < . 025 ). I*m not sure what to make 
of this finding. Evidently, very close proximity of the 
question and question related material is not a critical 
condition, and Ss are more likely to skip over material to 
get to something relevant than they are to skip over 
material in a paragraph after they have read something rele- 
vant to a question. 

Finally, the mode of questioning (multiple-choice 
or constructed response) did not influence learning. 

Conclusion 

It may very well be true, as some authors have 
claimed, that mature readers can learn as much from prose 
as from programmed materials. But, as we have seen, it is 
also possible to Improve learning from prose materials by 
using fairly simple stimulus controls - such as questions. 
Programming learning from prose materials then, requires a 
slight modification of technique. There are analogies 
between the pacing condition in the present study, and the 
step- size variable in conventional programs, and between 
the position of questions and prompting and confirmation as 
used in programming, which bear closer analysis. The fact 
that learning from gross instructional materials can be 
manipulated in a predictable manner suggests that the tech- 
nology of instruction will soon Include an understanding 
and application of relatively flexible methods of control. 



The present results have some practical implica- 
tions. Reflecting .upon current textbaok practice reveals 
that most authors tend to place questions at the end of 
chapters. In terms of the present study this is not an 
optimal strategy. Or rather^ it doesn't make much differ- 
ence where you put the questions in relation to the rele- 
vant content unless the questions are interspersed in the 
text immediately after relat:' vely small chunks of material.. 

As an adjunct to current texts these questions 
might he indicated by numbers in the margins of the text, 
the numbers retferrdng to the .ad.j.unct questions which are 
physically separate from the hook itself. The teacher or 
publisher could modify these questions very easily. 

At any rate, there are practical^ as well as 
'theoretical implications of the present research, which I 
think make an interesting hut relatively unexplored topic,. 



APPENDIX 



Sub.jects 

128 introductory psychology students. 

Design 

2x4x2x2x 2 analysis of variance with repeated 
measures on the last factor. 

Factor 

1. Question location (before or after paragraphs) 
-2 levels 

2. Question pacing (1 question after every 10 
sentences, 2 questions after every 20 sentences 

4 questions after every 40 sentences, or 

5 questions after every 50 sentences) -4 levels. 

3. Content location (question-relevant content 
located In the 1st or 2nd part of each 10 
sentence paragraph) -2 levels. 

4. Question mode (multiple-choice or constructed 
response) -2 levels. 

5. Posttest Items (covering prose content which 
was relevant or Incidental to the questions 
which Ss viewed along with the text material) 
-2 levels. 

Prose Material 



2000 word passage about William James. 
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Tesfe Questions' 



Same factual- knowledge questions used on posttest 
as in text. Relevant and incidental questions were of equal 

f>tr\ , . 
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while reading the text, but his posttest would Include Items 
1>. 2,, 3,. 4,. 5,, 6, etc. Items 2, 4, 6 would thus test his 
incidental learning.. Relevant and; Incidental subsets were 

counterbalance.d. across, groups. 

Constructed response items, were created by. dropping 
the alternative from the multiple-choice items and inserting 
a’ blank in the stem of the; question. 

Procedure 

Sub4;^bs were - allowed, to read the prose at. their, 
own pace. Questions were Interspersed in the prose according 
to experitaentai conditions.,. No) knowledge of results were 
given after the question. Subjects were not allowed to turn, 
back- once, they had finished; reading a= pag^ of text . After 
reading, ^^went on to; the 40 item posttest. 



RETENTION 







SENTENCES BETWEEN QUESTIONS 



Fig. 1. Retention as a function of question position and pacing. 
C=average of all conditions# 
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SENTENCES BETWEEN . QUESTIONS 



Fig, 2, Retention as a function of retention item and pacing, 
C=average of all conditions. 
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RETENTION 




Fig. 3. Retention as a function of retention item and position 
of content within each 10 sentence paragraph. C=average of all 
conditions* 



